Imaging device and information acquisition system in which an acquired image and associated information are held on a display

ABSTRACT

An imaging device includes an imaging portion that images a subject; a positional information acquisition portion that acquires positional information of an imaging position; a control portion which acquires information on the subject based on the positional information, and displays image data of the subject and the information on the subject on a display portion; and a hold control portion that outputs a hold control signal, which holds the image data of the subject and the information on the subject, to the control portion.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.16/656,062, filed Oct. 17, 2019, pending, which is a divisional of U.S.application Ser. No. 15/668,032, filed Aug. 3, 2017, now U.S. Pat. No.10,452,914, which is a divisional of U.S. application Ser. No.15/207,228, filed Jul. 11, 2016, now U.S. Pat. No. 9,756,253, which is adivisional of U.S. application Ser. No. 13/021,198, filed Feb. 4, 2011,now U.S. Pat. No. 9,420,251, which is a non-provisional applicationclaiming priority to and the benefit of U.S. provisional application No.61/306,171, filed Feb. 19, 2010 and Japanese Patent Application No.2010-025998, filed Feb. 8, 2010, the entire contents of which areincorporated herein by reference.

FIELD OF THE INVENTION

The present invention relates to an imaging device which has a functionof acquiring information of a position where a subject is imaged uponimaging the subject, thereby making the subject correspond to thepositional information; an imaging device, an information acquisitionsystem, and a program which acquire the information of the subject usingthe information.

DESCRIPTION OF THE RELATED ART

Conventionally, in portable information appliances such as imagingdevices (for example, digital cameras) having an imaging function or thelike, there are appliances which have a function of detecting positionalinformation where the portable information appliance is situated,thereby processing the image data which is imaged so as to correspond tothe detected positional information (for example, see JP-A-2004-15187).

However, for example, in such the above Patent Document, it isimpossible for a user to easily obtain the information on a subject(e.g., a structure) which is imaged by a user.

An object according to aspects of the present invention is to provide animaging device, an information acquisition system, and a program thatenables a user activity to be enriched.

SUMMARY

An imaging device according to an aspect of the present inventionincludes an imaging portion which images a subject; a positionalinformation acquisition portion which acquires positional information ofan imaging position; a control portion which acquires information on thesubject based on the positional information and displays image data ofthe subject and information on the subject on a display portion; and ahold control portion that outputs a hold control signal, which holds theimage data of the subject and the information on the subject, to thecontrol portion.

An imaging device according to an aspect of the present inventionincludes a latitude and longitude detection portion that detectslatitude and longitude information of one's position; an azimuthal angledetection portion which detects an azimuthal angle by which image datais imaged; a control portion which acquires augmented realityinformation added to a structure situated in a latitude and longituderange of the azimuthal angle direction in the latitude and longitudeinformation by the latitude and longitude information and the azimuthalangle, thereby displaying the augmented reality information on a displayportion; and a hold control portion which outputs the hold controlsignal, which stores the augmented reality information and the imagedata in a memory portion, to the control portion upon detecting anaction of storing the augmented reality information in the memoryportion.

An information acquisition system according to an aspect of the presentinvention comprises an imaging device and an information retrievalsystem. The imaging device is the imaging device according to any one ofthe above-mentioned image devices.

An information acquisition system according to an aspect of the presentinvention is constituted by an imaging device and an informationretrieval system, where the information retrieval system extracts astructure situated in a latitude and longitude range of a direction ofthe azimuthal angle in the latitude and longitude information by thelatitude and longitude information and the azimuthal angle transmittedfrom the imaging device, thereby transmits the information added to theextracted structure to the imaging device. The imaging device includes alatitude and longitude detection portion which detects latitude andlongitude information of one's position; an azimuthal angle detectionportion which detects an azimuthal angle by which image data is imaged;a control portion which acquires augmented reality information added toa structure situated in a latitude and longitude range of the directionof the azimuthal angle in the latitude and longitude information by thelatitude and longitude information and the azimuthal angle, therebydisplays the same on a display portion; and a hold control portion whichoutputs the hold control signal, which stores the augmented realityinformation and the image data in a memory portion, to the controlportion upon detecting an action that stores the augmented realityinformation in the memory portion. The information retrieval systemincludes a database in which map data, in which a structureidentification number of a structure corresponds to latitude andlongitude information of the structure, and a structure table, in whichthe structure identification number corresponds to the augmented realityinformation of the structure shown by the structure identificationnumber, are stored; and an information retrieval server which retrievesthe structure identification number of the structure situated in thelatitude and longitude range of the direction of the azimuthal angle inthe latitude and longitude information from the map data by the latitudeand longitude information and the azimuthal angle transmitted from theimaging device, reads the augmented reality information added to thestructure shown by the structure identification number from thestructure table by the retrieved structure identification number, andtransmits the augmented reality information of the read structure to theimaging device.

A program according to an aspect of the present invention is a programfor executing the function of the imaging device described in any one ofthe above in a computer and comprises a step of inputting a positionalinformation of a position having imaged a subject; a step of acquiringan information for the subject based on the positional information; astep of displaying an image data of the subject and the information forthe subject on a display portion; and a step of outputting a controlsignal holding the image data of the subject and the information for thesubject to the control portion.

A program according to an aspect of the present invention is a programfor executing the function of the imaging device described in any one ofthe above in a computer, a step of inputting latitude and longitudeinformation of one's position which is detected by a latitude andlongitude detection portion; a step of inputting an azimuthal angleimaging image data which is detected by an azimuthal angle detectionportion; a step of acquiring augmented reality information added to astructure situated in a latitude and longitude range of the azimuthalangle direction in the latitude and longitude information by thelatitude and longitude information and the azimuthal angle, therebydisplaying the same on a display portion; and a step of outputting ahold control signal, which stores the augmented reality information andthe image data in the memory portion, to the control portion upondetecting an action of storing the augmented reality information in amemory portion.

According to the aspects of the present invention, an imaging device, aninformation acquisition system and a program are provided that enables auser to be enriched.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram that shows a configuration example of aninformation acquisition system according to an embodiment of the presentinvention.

FIG. 2 is a conceptional diagram that shows a configuration example of astructure table stored in a database in FIG. 1.

FIG. 3 is a flowchart that shows a flow of processing informationretrieval of an image that is imaged by a digital camera.

FIG. 4 is a conceptional diagram that shows a display image displayed ona digital camera.

FIG. 5 is a conceptional diagram that shows a display image displayed ona digital camera.

FIG. 6 is a conceptional diagram that shows a display image displayed ona digital camera.

FIG. 7 is a flowchart that shows a flow of processing of an informationretrieval of an image after being imaged by a digital camera.

FIG. 8 is a flowchart that shows a flow of processing of an informationretrieval from a shop information input to a digital camera.

FIG. 9 is a block diagram that shows an information acquisition systemhaving a function of acquiring and storing an AR (Augmented Reality)information.

FIG. 10 is a diagram that shows a composition image in a display portionof a real image of buildings and images of tags (imaginary description)on which each of the augmented reality information (AR information) ofthe buildings is described.

FIG. 11 is a diagram that shows an AR information table stored in an ARinformation memory portion in FIG. 9.

FIG. 12 is a diagram that shows a retrieval range which retrieves the ARinformation of the structure.

FIG. 13 is a flowchart that shows an operational example of theinformation retrieval system in FIG. 9.

DESCRIPTION

Hereinafter, an imaging device and an information acquisition systemaccording to an embodiment of the present invention will be describedwith reference to the drawings. FIG. 1 is a block diagram that shows aconfiguration example of an imaging device and an informationacquisition system according to the present embodiment.

In FIG. 1, the information acquisition system includes a digital camera1 which is an example of an imaging device, and an information retrievalsystem 2. The digital camera 1 performs the transmission and thereception of the data to the information retrieval system 2 (or aninformation retrieval server 21 described later) via a wireless basestation 3 and an information communication network 4 such as theInternet by performing data communication with the wireless base station3 by radio communication. In addition, the imaging device in the presentembodiment is a portable terminal having an imaging function such as theaforementioned digital camera 1 or a mobile phone equipped with acamera. For example, the digital camera 1 has a browser function and hasa control portion 11, a transmission and reception portion 12, animaging portion 13, a GPS (Global Positioning System) 14, a memoryportion 15, an azimuth sensor 16 (an azimuthal angle detection portion),a display portion 17, and a timer 18. Herein, the radio communication inthe present embodiment does not use a line as a transmission line, butincludes communication by radio wave, light, sound wave or the like.

The imaging portion 13 has a CCD (Charge-Coupled Device) or CMOS(Complementary Metal Oxide Semiconductor) image sensor, a lens or thelike and outputs image data, which images the subject (a target to beimaged), to the control portion 11. In the following description, as anexample, the subject in the present embodiment shows structures(commercial buildings including stores, public buildings such as schoolsand hospitals, factories, houses including apartments, artificialconstructions such as steel towers, bridges, dams, amusement parks, andbronze statues, or the like). The GPS 14 (the positional informationacquisition portion, the latitude and longitude detection portion)acquires the information of the latitude and the longitude (the latitudeand longitude information, and the positional information) of thedigital camera 1 and outputs the same to the control portion 11 when thecontrol signal (for example, the control signal that is output by aperipheral circuit (not shown) which detects the push-down of an imagingbutton B1) instructing the imaging from a user, or the control signal(e.g., the control signal that is output by a peripheral circuit (notshown) which detects the selection of an icon by touching the icon (asign) showing the retrieval or the like by a user) instructing that theinformation retrieval is input. In addition, the positional informationacquisition portion (the latitude and longitude detection portion) inthe present embodiment may be another configuration if the position ofthe imaging device 13 can be acquired and, for example, may acquire thepositional information of the imaging device 13 using the wireless basestation 3.

The azimuth sensor 16 (the azimuthal angle detection portion, theazimuthal information acquisition portion) includes an electroniccompass or the like, detects the azimuth of the series direction wherethe CCD and the lens are arranged, for example, in series in the imagingportion 13 and outputs the same as azimuthal information to the controlportion 11 when the control signal (e.g., the control signal that isoutput by a peripheral circuit (not shown) which detects the push-downof an imaging button B1) instructing the imaging from a user, or thecontrol signal (e.g., the control signal that is output by a peripheralcircuit (not shown) which detects the selection of an icon by touchingthe icon showing the retrieval or the like by a user) instructing theinformation retrieval is input. In addition, the azimuth sensor 16 maydetect the azimuth of the direction (in other words, the imagingdirection) of the subject viewed by a user and outputs the same to thecontrol portion 11 as the azimuthal information. Herein, when theazimuthal angle is adopted as the azimuthal information, the azimuthalangles are represented as east (90°), south (180°) and west (270°) in aclockwise direction based on north (0°) as units of degrees, minutes,and seconds, for example, by setting the latitude and the longitude ofthe positional information as the center point.

The control portion 11 gives the captured image data the imageidentification information when the control signal instructing thecapturing such as pressing down the imaging button B1 by a user is inputfrom a peripheral circuit (not shown), allows the image data imaged foreach image identification information to correspond to the time data,the positional information and the azimuthal information acquired fromthe timer 18 and writes them on the memory portion 15 in the imagingorder.

The display portion 17 displays the image data which is imaged by theimaging portion 13 or the image data that is selected as an imagedisplayed by a user and is read from the memory portion 15 by thecontrol portion 11.

Furthermore, as described later, the display portion 17 performs thedisplay of the image data received from the information retrieval system2 or the character data input by a user or the like, by the control ofthe control portion 11.

The transmission and reception portion 12 performs the transmission andthe reception to the wireless base station 3 by the radio communicationand performs the transmission and the reception of the image data, thecharacter data or the data such as the control signal between it and theinformation retrieval system 2 via the information communication network4.

Next, as shown in FIG. 1, the information retrieval system 2 has aninformation retrieval server 21 and a database 22. In addition, thedatabase 22 may be provided in a memory medium (e.g., a memory, an HDDor the like) in the information retrieval server 21 and may be in anexternally attached memory medium or a memory medium of anotherterminal.

As in the structure table shown in FIG. 2, in the database 22, astructure identification information for identifying the structure, astructure name that is a name of a structure, a structure information(information such as an address, a phone number, a type, and aperipheral image data around the structure), positional information bythe latitude and the longitude of the structure, a structure description(information described by a store in the case of a store), and asubmission information (a comment such as an evaluation of a visiteduser, image data submitted by a user or the like) are stored.

Furthermore, in the database 22, a map data is stored in which thelatitude and the longitude are set as coordinate axis of atwo-dimensional plane and the respective registered structures arearranged. In the map data, the respective structures are shown in thepositions of the latitude and the longitude corresponding to thestructure identification information.

The information retrieval server 21 explores the structure which is thenearest distance in a direction where the azimuthal information showsfrom a position of the latitude and the longitude which the positionalinformation shows using the positional information and the azimuthalinformation to be input, and acquires the structure identificationinformation of the explored structure.

Furthermore, the information retrieval server 21 retrieves the structurecorresponding to the structure identification information of theexplored and acquired structure from the structure table and transmitsthe respective information (the structure name, the structureinformation etc.) of the explored structure to the digital camera 1. Inaddition, the information retrieval server 21 may selectively transmiteach piece of information of the structure to be transmitted to thedigital camera 1, for example, depending on necessity such as thetransmission and the reception data capacity. In that case, theinformation retrieval server 21 may transmit the rest of the informationamong the respective information of the structure to the digital camera1 again by a predetermined operation (e.g., the request from the digitalcamera 1) or the like.

<Information Retrieval of Image Imaged by Digital Camera 1>

Next, the operation of the present embodiment will be described usingFIGS. 1, 3, and 4 to 6. FIG. 3 is a flowchart which shows an operationalexample of the information retrieval processing in the presentembodiment of the case of performing the information retrieval in thestate of imaging the digital camera 1 toward the structure. FIGS. 4 to 6are diagrams that show arrangements of the screen displayed on thedisplay portion 17 of the digital camera 1 and the imaging button B1 ofthe digital camera 1.

In the following description, for example, a state will be described inwhich a user explores a restaurant for dinner while walking alongdowntown.

A user directs the lens of the digital camera 1 to a restaurant of anappealing exterior while wandering downtown (step S1), when a restaurantof a favorite exterior in which an information wants to be obtained isfound, as shown in FIG. 4, by touching (pressing down) a browse icon I1provided in an image display column MP of the digital camera 1, thebrowse icon I1 is selected (step S2).

At this time, as shown in FIG. 4, in the imaging state, the controlportion 11 displays the image data imaged by the imaging portion 13which is input from the imaging portion 13, the image data of therestaurant, in the image display column MP via the display portion 17.In addition, the image display column MP (the image display portion) inthe present embodiment is constituted by a touch panel (e.g., includinga pressure sensitive type, an electrostatic type, a sound wave type orthe like).

In addition, the above-mentioned step S2 may be configured so that,after pressing down the imaging button B1 of the digital camera 1, thebrowse icon I1 is touched and selected while viewing the image displaycolumn MP on which the image of the imaged restaurant is displayed.

The browse icon I1 is touched, whereby the signal, which shows that thebrowse icon I1 is selected, from a peripheral circuit (not shown) to thecontrol portion 11 is output. Moreover, the control portion 11 detectsthe selection of the browse icon I1 by the inputting signal showing thatthe browse icon I1 is selected, gives the image data of the restaurantthat is imaged by the imaging portion 13 the image identificationinformation, allows the image data to correspond to the imageidentification information, writes the image data and the imaged timefor the memory portion 15, and stores the image data (step S3).

Furthermore, when the browse icon I1 is touched, the control portion 11allows the positional information acquired by the GPS 14 and theazimuthal information acquired by the azimuth sensor 16 to correspond tothe image identification information as the related information, writesand stores them on the memory portion 15 (step S4).

Moreover, after writing the image data and the related information onthe memory portion 15, the control portion 11 transmits a retrievalrequest signal including a camera identification information (anidentification information given to a user or an identificationinformation given to the digital camera 1) for identifying the digitalcamera 1, the positional information, and the azimuthal information tothe information retrieval system 2 via the transmission and receptionportion 12 (step S5). In addition, the control portion 11 may transmitthe retrieval request signal to the information retrieval system 2 viathe transmission and reception portion 12 without writing the imagedata, the related information or the like on the memory portion 15 inthe above-mentioned steps S3 to S5.

Next, in the information retrieval system 2, when the retrieval requestsignal is received from the digital camera 1, the server 21 extracts thestructure of the retrieval target from the map data of the database 22based on the positional information and the azimuthal informationincluded in the retrieval request signal. Moreover, the server 21transmits the respective information in the table of the structure fromthe structure table of FIG. 2 by the structure identificationinformation to the digital camera 1 as the retrieval result informationincluding the structure identification information (step S7). At thistime, as an example, the server 21 reads the address on the network ofthe digital camera 1 which is added when the retrieval request signal istransmitted from the digital camera 1 via the transmission and receptionportion 12, and transmits the retrieval result information to theaddress.

In addition, in the above-mentioned step S7, the server 21 may extractthe structure of the retrieval target from the map data of the database22 based on the positional information and the azimuthal informationincluded in the retrieval request signal, only when it is detected thatthe camera identification information included in the retrieval requestsignal is registered in a user registration table in the userregistration table stored in the database 22. In this case, the server21 may transmit, for example, the information on the need for the userregistration based on the camera identification information to thedigital camera 1, when it is detected that the camera identificationinformation included in the retrieval request signal is not registeredin the user registration table.

Moreover, in the digital camera 1, when the retrieval result informationis received from the information retrieval system 2 via the transmissionand reception portion 12, as shown in FIG. 5, the control portion 11displays the building information on the information display column SPvia the display portion 17 (step S8). For example, when the structure isa restaurant, information on the kind (Chinese, Japanese, French,Italian or the like) of restaurant, a phone number, an address of therestaurant or the like are displayed. In addition, the informationdisplay column SP shown in FIG. 5 is an example of the presentembodiment, and a part of the information display column SP may bedisplayed on the image of the restaurant so as to be overlapped.Furthermore, the information display column SP is a part of theabove-mentioned image display column MP and is displayed due to, forexample, the above-mentioned browse icon I1, a CM icon I2 describedlater or the like is touched and selected.

Furthermore, when a user touches the CM icon I2, the peripheral circuitoutputs the signal showing the section of the CM icon I2 to the controlportion 11.

As a result, the control portion 11 detects the selection of the CM iconI2 by the input of the signal showing that the CM icon I2 is selected,and displays the commercial information (a menu, a seat number, an imagein a store, today's recommendation dish of a chef or the like) of therestaurant contained in the retrieval result information on theinformation display column SP via the display portion 17. Furthermore,when the image data such as the image in the store is included in thecommercial information, the control portion 11 displays the image dataon the information display column SP.

Furthermore, when a user touches the submission reading icon I3, theperipheral circuit outputs the signal showing the selection of thesubmission reading icon I3 to the control portion 11.

As a result, the control portion 11 detects the selection of thesubmission reading icon I3 by the input of the signal showing that thesubmission reading icon I3 is selected, and displays the submissioninformation (the comment) written by another user included in theretrieval result information on the information display column SP viathe display portion 17. Moreover, when a plurality of submissioninformation exists, the control portion 11 displays the plurality ofsubmission information on the information display column SP, forexample, in the submitted date order.

The image data is also included in the submission information, and, whenthe photographs of the respective dish imaged by another user or theinternal image data of the store is included in the submissioninformation, the control portion 11 displays the image data on theinformation display column SP via the display portion 17.

Moreover, for example, when the control portion 11 detects the signalshowing that it is touched so as to be slid from the left end of theinformation display column SP toward the right direction in FIG. 5, thecontrol portion 11 ends the display of the information display column SPand displays the image of the restaurant on the image display column MPvia the display portion 17 as shown in FIG. 4. In addition, for example,when the control portion 11 detects the signal showing that thesubmission reading icon I3 is selected while the submission informationis displayed on the information display column SP, the control portion11 may end the display of the information display column SP or thedisplay of the submission information.

Furthermore, for example, when displaying the commercial information (orthe submission information) after displaying the building information onthe information display column SP, the control portion 11 may displaythe commercial information simultaneously with the building informationand may display the commercial information after erasing the buildinginformation from the information display column SP.

In addition, when the control portion 11 detects the signal showing thatthe image in the store or the like displayed on the information displaycolumn SP is touched, the control portion 11 displays the expansionimage or the reduction image of the image in the store on theinformation display column SP (or the image display column MP).

Moreover, when a user touches a mail icon I4, the peripheral circuitoutputs the signal showing the selection of the mail icon I4 to thecontrol portion 11.

As a result, the control portion 11 detects the selection of the mailicon I4 by the input of the signal showing that the mail icon I4 isselected, and displays an input column IP shown in FIG. 6 on a part ofthe image display column MP via the display portion 17. Herein, theinput column IP includes a text column T into which a user inputs theinformation or the like, and a touch screen type keyboard portion K. Theinput column IP is used when a user inputs submission informationdescribed later, a store name, a year or the like.

A user writes the submission information such as the dish they eat, theambience in the store or the quality of the service of a waiter on thetext column T of the input column IP using the touch screen typekeyboard portion K. Moreover, a user touches the mail icon I4 again,whereby the control portion 11 detects the section of the mail icon I4,sets the character information described in the text column T and theimage data displayed on the image display column P as the submissioninformation together with the camera identification information and thestructure identification information, and transmits the retrievalrequest signal including the submission information to the informationretrieval system 2 via the transmission and reception portion 12.

Moreover, when receiving the retrieval request signal from the digitalcamera 1, the server 21 writes new submission information, which isinput so as to correspond to the structure identification information,with respect to the submission column of the structure table of thedatabase 22. In addition, before the submission information, which waswritten on the submission writing column (the text column T), is writtenon the submission column of the structure table of the database 22, theserver 21 detects whether or not the camera identification informationincluded in the retrieval request signal exists in the user registrationtable in the database 22, and only when it is detected that the cameraidentification information is registered, the submission information maybe written on the submission column of the structure table of thedatabase 22.

Next, the control portion 11 performs the detection whether or not anending icon E is selected (step S9). Herein, when a user touches theending icon E, the peripheral circuit outputs the signal showing theselection of the ending icon E to the control portion 11.

Moreover, if the control portion 11 detects the selection of the endingicon E by the input of the signal showing that the ending icon E isselected, then the control portion 11 ends the processing of theinformation retrieval.

Meanwhile, when the signal showing the selection of the ending icon E isnot input and the control portion 11 does not detect the selection ofthe ending icon E, the control portion 11 returns the processing to stepS1 and continues the imaging processing and the information retrievalprocessing of the digital camera 1.

Furthermore, when the browse icon I1 is selected, the control portion 11may include the captured image data with respect to the retrievalrequest signal to be transmitted to the information retrieval system 2.Moreover, the server 21 compares the peripheral image data of thestructure information retrieved from the structure table by thestructure identification information of the structure extracted from themap data with the transmitted image data by the image identification,extracts the characteristic amount from the image data, compares theextracted amount with the characteristic amount of the stored peripheralimage data, and may perform the determination whether or not itcorresponds to the imaged structure by the detection whether or not thesimilarity is equal to or greater than a set numerical value.

At this time, when it is decided that, as the comparison result, thestructure of the peripheral image data is not the same as the structureof the image data, the server 21 extracts the next structure, which isclosest in distance, from the map data by the positional information andthe azimuthal information, performs the comparison of the characteristicamounts again, and may perform the evaluation of the similarity betweenthe capturing image data and the newly extracted peripheral image dataof the structure.

In addition, similarly to the above-mentioned imaging button B1, thebrowse icon I1 in the present embodiment may be provided in the imagingdevice 1 as a browse button (not shown). In this case, the function ofthe browse button is the same as that of the above-mentioned browse iconI1. Furthermore, the imaging button B1 in the present embodiment mayfunction as the browse button by operating a switch portion (not shown).In this case, the browse icon I1 or the browse button is unnecessary.

<Image Information Retrieval After Imaging By Digital Camera 1>

Next, the operation of the present embodiment will be described usingFIGS. 1, 7 and 4 to 6. FIG. 7 is a flowchart which shows an operationalexample of the information retrieval processing in the presentembodiment in a case where, after the structure is imaged by the digitalcamera 1, the image data stored in the memory portion 15 is read and theinformation retrieval relative to the image data is performed.

In the following description, for example, an example is shown whichshows a state in which a user retrieves a restaurant for dinner whilewandering downtown Shinjuku, images the image data of some restaurants,and then finds the restaurant for dinner from amongst them.

A user directs the digital camera 1 with respect to the restaurant of anappealing exterior while wandering downtown (step S11), and when hefinds the restaurant of the exterior desired to perform the informationretrieval later, he presses down the imaging button B1 (the shutter)provided in the digital camera 1 (step S12). Herein, the control portion11 displays the image data of the restaurant which images the capturingimage data of the imaging portion 13 on the image display column MP ofthe display portion 17 as shown in FIG. 4.

Next, due to the imaging button B1 is pressed down by a user, theperipheral circuit outputs the signal showing the press-down of theimaging button to the control portion 11.

Moreover, the control portion 11 detects the press-down of the imagingbutton B1 by the input of the signal showing that the imaging button B1is pressed down, gives the capturing image data of the restaurant of theimaging portion 13 the image identification information, allows theimage data to correspond to the image identification information, andwrites and stores the image data and the imaged time on the memoryportion 15 (step S13).

Moreover, the control portion 11 allows the positional informationacquired by the GPS 14 and the azimuthal information acquired by theazimuth sensor 16 to correspond to the image identification informationat a point in time when the imaging button B1 is pressed down, andwrites and stores the information on the memory portion 15 (step S14).

Next, when a user performs the information collection of the imagedrestaurant, the user touches the reading icon R of the captured image.As a result, the peripheral circuit outputs the signal showing theselection of the reading icon R to the control portion 11. Herein, thecontrol portion 11 performs the decision whether or not the image isstored in the memory portion 15 depending on whether or not the signalshowing the selection of the reading icon R is input (step S15).

At this time, when the signal showing the selection of the reading iconR is input, the control portion 11 detects the selection of the readingicon R and progresses the processing to step S16 so as to start theretrieval of the information. In addition, the reading icon R may be areading button (not shown). In that case, the reading button may beprovided in the digital camera 1 main body, similarly to the imagingbutton B1. Meanwhile when the signal showing the selection of thereading icon R is not input or when the image is not stored in thememory portion 15, the control portion 11 returns the processing to thestate in step S11 so as to perform a new imaging processing.

Moreover, when a user touches the reading icon R and the signal showingthe selection of the reading icon R is input from the peripheralcircuit, the control portion 11 sequentially reads the image data in asequence which is imaged from the memory portion 15 in time series (stepS16), and displays the image data on the image display column MP via thedisplay portion 17 for each preset sheet (or for each image data) as athumbnail image.

Furthermore, in a case where the thumbnail image does not completelyenter one page (or in the case of displaying one by one), when a usertouches the image displayed on the image display column MP so as to beslid in a predetermined direction, the thumbnail image of the prior pageor the next page is displayed on the image display column MP. At thistime, the control portion 11 displays the thumbnail image of the priorpage or the next image on the image display column MP via the displayportion 17 by detecting the signal that the image displayed on the imagedisplay column MP is touched so as to be slid in a predetermineddirection.

Next, for example, when a user selects the image of the favoriterestaurant from the thumbnail image by touching and touches the browseicon I1, the peripheral circuit outputs the signal showing the selectionof the browse icon I1 together with the selected image identificationinformation to the control portion 11.

As a result, when the signal showing the press-down of the browse iconI1 is input, the control portion 11 reads the positional information andthe azimuthal information corresponding to the image identificationinformation from the memory portion 15 by the image identificationinformation of the selected image data, and transmits the retrievalrequest signal including the camera identification information, thepositional information, and the azimuthal information to the informationretrieval system 2 via the transmission and reception portion 12 (stepS17).

Since the following processing of step S6 to step S8 are the same asthat of step S6 to step S8 in FIG. 3, the description thereof will beomitted. Next, the control portion 11 performs the detection of whetheror not the ending icon E is selected (step S18). At this time, when auser touches the ending icon E and the signal showing the selection ofthe ending icon E is input from the peripheral circuit, the controlportion 11 finishes the processing of the browsing. Meanwhile, when thesignal showing the selection of the ending icon E is not input, thecontrol portion 11 returns the processing to step S16, and continues theselection processing of the image from the thumbnail image.

<Information Retrieval from Store Information Input in Digital Camera 1>

Next, the operation of the present embodiment will be described usingFIGS. 1, 8 and 4 to 6. FIG. 8 is a flowchart which shows an operationalexample of the information retrieval processing in the presentembodiment in a case where a user wants to perform the informationretrieval of the restaurant or the like of the input store name from thestructure table of FIG. 2 of the corresponding store name stored in thedatabase 22, by inputting the store name in the digital camera 1.

In the following description, a state is shown in which, for example, inthe case of having dinner in Harajuku, the information of the restaurantheard from an acquaintance is confirmed. When a user selects theretrieval icon I5 by touching, the peripheral circuit outputs the signalshowing the selection of the retrieval icon I5 to the control portion11.

As a result, the control portion 11 detects the selection of theretrieval icon I5 by the input of the signal showing that the retrievalicon I5 is selected, and displays the input column IP shown in FIG. 6 ona part of the image display column MP via the display portion 17.

Moreover, a user writes the store name of the restaurant desired toretrieve using the touch screen type keyboard portion K to the textcolumn T of the input column IP (step S21), and the user touches theretrieval icon I5 again (step S22).

As a result, the peripheral circuit outputs the character data of thestore name input in the text column T together with the signal showingthe selection of the retrieval icon I5 to the control portion 11.Moreover, the control portion 11 detects the selection of the retrievalicon I5 by the input of the signal showing the selection of theretrieval icon I5, reads the character data input in the text column T,and transmits the retrieval request signal together with its cameraidentification information as the retrieval request signal to theinformation retrieval system 2 via the transmission and receptionportion 12 (step S23).

Next, when the retrieval request signal is input, the server 21 readsthe information (the store information, each information in the table ofFIG. 2) of the structure (the store) corresponding to the store namefrom the structure table in the database 22 (step S24).

After reading the store information, the server 21 transmits theacquired store information to the digital camera 1 (step S25).

When the store information is received via the transmission andreception portion 12, the control portion 11 displays the image dataaround the store, which is included in the structure information, on theimage display column MP of FIG. 5 via the display portion 17, anddisplays the building information on the information display column SP(step S26). For example, information on the type (Chinese, Japanese,French, Italian or the like) of the restaurant, the phone number or thelike is displayed.

Furthermore, when a user selects the CM icon I2 by touching, the controlportion 11 detects the selection of the CM icon I2, and displays thecommercial information (a menu, today's recommended dish of a chef orthe like) of the restaurant included in the retrieval result informationon the information display column SP via the display portion 17.

Furthermore, when a user selects the submission reading icon I3 bytouching, the peripheral circuit outputs the signal showing theselection of the submission reading icon I3 to the control portion 11.

The control portion 11 detects the selection of the submission readingicon I3 by the input of the signal showing that the submission readingicon I3 is selected, and displays the submission information written byanother user included in the retrieval result information on theinformation display column SP via the display portion 17.

In addition, when a plurality of submission information exists, thecontrol portion 11 sequentially displays the plurality of submissioninformation on the information display column SP via the display portion17. Furthermore, when the image data is also included in the submissioninformation, the control portion 11 displays the image data on theinformation display column SP via the display portion 17.

Moreover, the control portion 11 performs the detection whether or notthe ending icon E is selected (step S27). Since the detection of theselection of the ending icon E is the same as the processing of step S9in <Information Retrieval of Image Imaged by Digital Camera 1> (in FIG.3), the description thereof will be omitted.

At this time, when the signal showing the selection of the ending icon Eis input from the peripheral circuit by touching the ending icon E by auser, the control portion 11 finishes the processing of the browsing.Meanwhile, when the signal showing the selection of the ending icon E isnot input, the control portion 11 returns the processing to step S21 andcontinues the retrieval of the information of the store.

<Charging Processing Relative to Store Registered in Database 22>

When the above-mentioned structure is a store (including a restaurant ora company), the structure table of the database 22 may have aconfiguration in which the data of a discount coupon is stored for eachstore.

Moreover, the server 21 adds the discount coupon information to theretrieval result information and transmits the same to the digitalcamera 1.

When a user has dinner or shops using the discount coupon information,the server 21 detects whether or not a user has dinner or shopsdepending on the use of the discount coupon information. For example,when the discount coupon information is a bar code (including atwo-dimensional bar code) or the like that is displayed on theinformation display column SP of the digital camera 1, a reader of astore reads the bar code and transmits the use information showing theuse of the discount coupon information together with the cameraidentification information, which is transmitted from the digital camera1, and a charge of the shopping (or a meal) to the information retrievalsystem 2.

As a result, when receiving the use information, the server 21 reads thehistory information corresponding to the camera identificationinformation added to the use information from the database 22, andperforms different charging with respect to the store used by a user inthe using step of the information whether he visits the store by theinformation retrieval, uses the store by viewing the informationprovided by the store, or uses the store by viewing the submissioninformation, that is, as the information providing charge depending onthe using degree of information of the database 22. Furthermore, theserver 21 writes the step in which the information retrieval isperformed, and the history of using the discount coupon information on ahistory table which is stored in the database 22 for each cameraidentification information.

The structure identification information showing a store is included inthe bar code. In the database 22, a charging table is provided whichstores the history of the charging corresponding to each store and theintegrated value of the charging corresponding to the structureidentification information.

Moreover, the server 21 corresponds to the structure identificationinformation, writes the used sum and the charging corresponding to theusing step of the information in the charging table as the history, andadds a new charging to the integrated value to perform the renewal ofthe integrated value.

<Addition Processing of Point Relative to User>

Whenever a user uses the discount coupon information in the store, theserver 21 obtains a point (e.g., calculated by multiplying the pointfactor by the using amount) depending on the use amount when using thediscount coupon information, and adds up the obtained points tocorrespond to the camera identification information in the userregistration table in the database 22.

Furthermore, the server 21 adds up the points of a preset value in theuser registration table to correspond to the camera identificationinformation with respect to a user who transmits the submissioninformation. The point can be used together with the discount couponinformation instead of money in the payment.

<Display Sequence of Submission Information>

Herein, the server 21 may transmit a nickname and a title as thesubmission information together with the submission identificationinformation in an initial step in the digital camera 1. Then, thecontrol portion 11 initially displays only a plurality of nicknames andtitles on the information display column SP. Then, a user touches andselects the appealing nickname and title submission information from theplurality of displays. As a result, the peripheral circuit transmits thesignal showing the selection of the nickname and the title and thesubmission identification information of the selected submissioninformation to the control portion 11.

As a result, the control portion 11 transmits the submissionidentification information and the submission information transmissionrequest to the information retrieval system 2.

As a result, the server 21 transmits the character data and the imagedata (the whole submission information) corresponding to the transmittedsubmission identification information to the digital camera 1.

Herein, the server 21 gives the respective submission information thesubmission identification information for the identification, allows theinformation to correspond to the camera identification information, andwrites and stores the information on the user registration table.

In addition, in the present embodiment, when the control portion 11receives the character data and the image data from the informationretrieval system 2 via the transmission and reception portion 12, thecontrol portion 11 may display the character data on the informationdisplay column SP and may display the image data on the image displaycolumn MP.

Furthermore, in the user registration table, the server 21 retrieves thesubmission identification information corresponding to the referencedsubmission information from the user registration tables and incrementsthe reference number of the camera identification informationcorresponding to the submission identification information.

Moreover, in the initial display step of the nickname and the titlewhich allows a user to select the submission information, the server 21transmits the display list to the digital camera 1 in a descending orderof the selection count of the submission information, that is, thedisplay list listed in a ranking order which the nickname and the titleof the user having a larger reference count is higher rank in displayingon the information display column SP.

As a result, the control portion 11 sequentially displays the nicknameand the title on the information display column SP according to thedisplay list.

<Display Processing of Past Image Data>

Furthermore, in the present embodiment, the database 22 may have theprevious image table in which the image data, in which the building andthe scene in each of the latitude and the longitude are imaged for eachyear, is stored to correspond to the latitude and the longitude.

When a user selects the previous image icon I6 by touching, theperipheral circuit outputs the signal showing the selection of theprevious image icon I6 to the control portion 11.

As a result, the control portion 11 detects the selection of theprevious image icon I6 by the input of the signal showing that theprevious image icon I6 is selected, and displays the input column IPshown in FIG. 6 on a part of the image display column MP via the displayportion 17.

Furthermore, a user writes the year (e.g., year of Christian Era) on thetext column T using a touch screen type keyboard portion K, and thentouches the previous image icon I6 (or the browse icon I1).

As a result, the peripheral circuit transmits the character data of theyear together with the signal showing the selection of the previousimage data I6 to the control portion 11. When the signal showing theselection of the previous image data I6 is detected, the control portion11 reads the character data showing the year that is written on the textcolumn T.

After reading the data of the year, the control portion 11 transmits theprevious image retrieval request together with the read year, thepositional information, the azimuthal information, and the cameraidentification information to the information retrieval system 2.

Next, in the information retrieval system 2, the server 21 selects theprevious image table corresponding to the latitude and the longitudebased on the positional information of the structure received from thedigital camera 1 displayed on the image display column P. Moreover, theserver 21 reads the azimuthal information in the selected previous imagetable and the image data corresponding to the year, and transmits theread image data to the digital camera 1. The digital camera 1 displaysthe image data corresponding to the year, which is received from theserver 21, on the image display column MP via the display portion 17. Inaddition, when there is the added information relative to the read imagedata , the server 21 also transmits that information together with theread image data to the digital camera 1.

As a result, a user can obtain the information on what structuresexisted for each past year or whether there was only a landscape or thelike.

Herein, when the previous image table of the corresponding latitude andthe longitude cannot be retrieved, the server 21 retrieves the previousimage table of the latitude and the longitude which is in a direction ofthe azimuthal information and is mostly near the corresponding latitudeand longitude, and performs the extraction of the image data therefrom.

Furthermore, if the year for retrieval is a period where photography didnot exist, a landscape drawn at that period or the image data of a CG(Computer Graphics) imaginarily made may be stored in the previous imagetable of the memory portion 15 to correspond to the year instead of theimage data of the photograph.

<Display Processing of AR (Augmented Reality) Information>

As mentioned above, in the configuration of FIG. 1, the control portion11 writes the latitude and longitude information (the positionalinformation) which shows the existence position of the digital camera 1itself obtained from the GPS 14, the azimuthal information including theazimuthal angle showing a direction (the imaging direction) of theoptical axis of the digital camera 1 obtained from the azimuth sensor 16(the azimuthal angle detection portion) or the like on the memoryportion 15 in an imaging sequence while adding the image identificationinformation to the image data in which the imaging portion 16 images thesubject.

As shown in FIG. 9, in the configuration of FIG. 1, as a function forthe acquisition and the memory of the augmented reality information (ARinformation), a hold control portion 31 and an AR information memoryportion 30 are newly added. FIG. 9 is a block diagram that shows aconfiguration example of an information acquisition system having afunction of acquiring and storing the AR information.

The AR information in the present embodiment shows various information(information on the subject) shown in the building table shown in FIG.2. For example, the AR information includes a structure identificationinformation for identifying the structure, a structure name that is aname of a structure, a structure information (information such as anaddress, a phone number, a type, and a peripheral image data around astructure), a positional information by the latitude, the longitude orthe like of the structure, a description of a structure (if a structureis a store, information described by the store), a submissioninformation (comment of the evaluation or the like of a visited user,the comment of a user, the submitted image data of the user or the like)or the like. Herein, the building identification information may be aURL that shows where the information of the building is stored.

That which the control portion 11 makes any one of the AR information asa tag (object of an augmented reality) which displays to be overlappedwith the image data of a through image (the image in which the subjectis imaged) may be configured so as to be arbitrarily set by a user.

Herein, the through image is image data in which the imaging portion 13continuously outputs the image imaged on the imaging element to thecontrol portion 11 as the image data and the control portion 11sequentially displays the same on the display portion 17.

The hold control portion 31 detects an action requiring the hold of theAR information. That is, the hold control portion 31 detects the timingof the hold which stores the image data, which is imaged on an elementsuch as a CCD by the imaging portion 13 and is displayed on the displayportion 17, and the AR information which is in the image data (which isin an angle of view of the image), on the AR information memory portion30. In addition, the AR information memory portion 30 may be provided inthe information retrieval system 2.

Herein, the control portion 13 stores the table showing thecorrespondence between the focus distance, the magnification or the likeand the angle of view on the memory portion 15 in advance and obtainsthe angle of view information of the image by the focus distance, themagnification or the like when acquiring the image data.

FIG. 10 is a diagram that shows a composition image (AR image) on thedisplay portion 17 of a real image (a through image) of the buildings T1to T4 and the image of the tags (e.g., the tag in which the structureinformation as the augmented reality is described) TB1, TB2, TB3 and TB4as the object of the augmented reality in which the respective ARinformation of the buildings T1 to T4 is described.

Furthermore, the digital camera 1 in the present embodiment has an ARinformation acquisition mode acquiring the AR information or a commonimaging mode. The control portion 11 performs a control which detectsthe press-down of the button by the signal from the switch or detectsthe touch of the icon on the display portion 17 by the touch sensor, bypressing down the button (the AR information acquisition button ARB1) ofthe AR information acquisition mode provided in the case of the digitalcamera 1 or touching the icon (the AR information acquisition icon ARI1)of the AR information acquisition mode displayed on the display portion17, thereby setting either the AR information acquisition mode or thecommon imaging mode.

Moreover, in the case of the AR information acquisition mode, thedigital camera 1 in the present embodiment has a mode (an AR displaymode) that displays (composites the image data and the object of theaugmented reality of the AR information) the AR information as the tagor a mode (an AR non-display mode) that sets as non-display (displayingonly the image data). The control portion 11 performs a control whichdetects the press-down of the button or the touch of the icon by thesensor, by the signal from the switch by the press-down of thedisplay/non-display button (the AR switching button ARB2) provided inthe case of the digital camera 1, or the touch of thedisplay/non-display button icon (the AR switching icon ARI2) displayedon the display portion 17, thereby setting any one of the display ornon-display.

Herein, in the display portion 17, a transparent touch sensor isprovided on the display element. When touching the icon, the touchsensor is touched, whereby the coordinate value of the touched area istransmitted to the control portion 11 as the detection signal. Thecontrol portion 11 matches the coordinate of the touch sensor with thecoordinate of the display element to be displayed and can decide (theicon displayed on the display portion 17 shows the starting of theapplication or identifies the image identification information of theimage data displayed as the thumbnail image or the like) the displayedimage by the coordinate value of the detection signal.

Whenever the azimuthal angle (the azimuthal information) of the digitalcamera 1 is changed, the control portion 11 transmits the latitude andlongitude information (the positional information), an azimuthalinformation and an angle of view information which image the image datadisplayed on the display portion 17 upon being held, together with theAR information acquisition request to the information retrieval system 2via the transmission and reception portion 12. In addition, the controlportion 11 may transmit the positional information and the ARinformation acquisition request to the information retrieval system 2via the transmission and reception portion 12 periodically or by theinstruction of a user and acquire the AR information. Furthermore, thecontrol portion 11 may transmit the image data and the AR informationacquisition request held by the hold control portion 31 to theinformation retrieval system 2 via the transmission and receptionportion 12 periodically or by the instruction of a user and acquire theAP information.

Herein, the control portion 11 detects the azimuthal angle for eachpredetermined period, and when the azimuthal angle is changed beyond apreset angle from the azimuthal angle, which is stored in the azimuthalangle memory portion of the memory portion 15 in advance, detects thechange as a change in azimuthal angle, and newly stores the azimuthalangle of this point in time in the azimuthal angle memory portion.

When the AR information acquisition request is received from the digitalcamera 1, the information retrieval server 21 obtains the retrievaldistance from the coordinate position of the latitude and the longitudeshown by the latitude and longitude information obtained from the angleof view information to the coordinate position of the latitude and thelongitude that retrieves the AR information, based on the latitude andlongitude information, the azimuthal information and the angle of viewinformation which are added to the AR information acquisition request.

Furthermore, the information retrieval server 21 obtains the retrievaldistance of the AR information by reading the retrieval distancecorresponding to the angle of view information from the retrievaldistance table of the database 22 in which the angle of view and theretrieval distance corresponding to the angel of view are described.

FIG. 12 is a diagram that shows a retrieval range that retrieves the ARinformation of the structure.

Moreover, the information retrieval server 21 obtains the retrievalrange shown in FIG. 12 by the coordinate position based on the latitudeand longitude information, the azimuthal angle of the optical axis ofthe lens of the digital camera 1, and the retrieval distance, and readsthe structure identification information, which is in the retrievalrange, from the map data of the database 22.

Furthermore, the information retrieval server 21 reads the ARinformation of the structure from the structure table in the database 22by the read structure identification information.

The information retrieval server 21 transmits the read AR informationtogether with the corresponding structure identification information tothe digital camera 1.

As a result, the control portion 11 matches the coordinate value basedon the latitude and longitude information of the digital camera 1 to thecoordinate value based on the latitude and longitude information shownby the latitude and longitude information included in the ARinformation, overlaps and composes the image of the tag (the object ofthe augmented reality) of the AR information with the image data (thethrough image), and displays the same on the display portion 17 in thestate shown in FIG. 10.

Furthermore, when the hold control signal is received from the holdcontrol portion 31, the control portion 11 performs the processing ofrecording the latitude and longitude information upon being held, theazimuthal information and the angle of view information, the ARinformation transmitted from the information retrieval system 2 by theAR information acquisition request, and the structure identificationinformation which identifies the structure of the AR information on theAR information memory portion 30 for each image identificationinformation of the image data upon being held.

FIG. 11 is a diagram that shows a configuration of the AR informationtable stored in the AR information memory portion 30 in FIG. 9. The ARinformation table is stored in the AR information memory portion 30 foreach piece of image identification information.

As described above, the control portion 11 is created for each imageidentification information, and writes and stores the structureidentification information (URL showing the place where the ARinformation of the structure is stored), the AR information (includingthe latitude and longitude information) of the structure, the latitudeand longitude information of the digital camera 1 itself when the holdcontrol signal is input and it is held, the azimuthal information, andthe angle of view information with respect to the AR information memoryportion 30 as the AR information table.

Furthermore, when the hold control signal is input, the control portion11 writes and stores the image data, which is imaged in the imagingelement of the imaging portion 13 at that time, on the memory portion 15to correspond to the image identification information given to the imagedata.

Herein, as an action of the input for instructing the hold of the ARinformation, the hold control portion 31 has a detection unit whichdetects the signal from the switch by the press-down of the ARinformation acquisition button ARB1 provided in the case of the digitalcamera 1, a detection unit by which the touch sensor detects thetouching of the icon (the AR information acquisition icon ARI1) showingthe AR information acquisition displayed on the display portion 17 orthe like.

Moreover, when the AR information acquisition button is pressed down, orwhen the icon showing the AR information acquisition is touched, thehold control portion 31 detects the input of the signal for recordingthe image data and the AR information, and outputs the hold controlsignal performing the hold control to the control portion 1. In thismanner, the hold control portion 31 detects the input of the image dataand the signal for recording the AR information by a motion (action,operation, movement) of a user with respect to the digital camera 1, andoutputs the hold control signal performing the hold control to thecontrol portion 1.

Furthermore, the hold control portion 31 may detect that the digitalcamera 1 is rapidly moved (in order to easily view the screen, thedigital camera 1 is directed downward and vibrated or the like) by anacceleration sensor as an action of holding the AR information, and setthe detection timing to the timing of the hold.

In the acceleration detection using the acceleration sensor, when theacceleration data of more than a predetermined threshold value issupplied from the acceleration sensor and the hold control portion 31outputs the hold control signal as the control signal of the holdinstruction to the control portion 11. In this manner, the hold controlportion 31 detects the input of the image data and the signal forrecording the AR information by the motion of a terminal (e.g., thedigital camera 1), and outputs the hold control signal performing thehold control to the control portion 1.

Furthermore, for example, as the action of holding the AR information,the hold control portion 31 detects the characteristic point of theimage and detects by whether the characteristic point moves in thepreset distance in the time of the preset detection range, or detects achange in image pattern in the image data by the change of the contrastof the output image data of the imaging element beyond a presetdifference, and sets the detection timing to the timing of the hold.

In the detection of the change in image pattern, when the change of theimage pattern, which is equal to or greater than the predeterminedthreshold value, is detected, the hold control portion 31 outputs thehold control signal as the control signal performing the hold control tothe control portion 11.

Moreover, for example, the hold control portion 31 may detect that anangle of the digital camera 1 is rapidly changed (moved by the operationthat, in order to easily view the screen, the digital camera 1 isdirected downward or moved by the vibration or the like) using anelevation sensor as the action of holding the AR information and set thedetection timing to the timing of the hold.

In the detection of the elevation information using the elevationsensor, when the elevation change, which is equal to or greater than thepreset threshold value, is supplied from the elevation sensor, the holdcontrol portion 31 outputs the hold control signal as the control signalperforming the hold control to the control portion 11. In this manner,the hold control portion 31 detects the input of the signal forrecording the image data and the AR information by the motion of theterminal (e.g., the digital camera 1) and outputs the hold controlsignal performing the hold control to the control portion 11.

Furthermore, in the case of the action of instructing of the hold usingthe above-mentioned acceleration sensor, a change in image pattern andthe elevation sensor, in order to acquire the AR informationcorresponding to the image data when the hold control signal isreceived, the control portion 11 needs to store the image data obtainedfor each AR information acquisition request, the structureidentification information corresponding to the image data, the ARinformation of the structure, the latitude and longitude information,the azimuthal information, and the angle of view information in thebuffer portion of the AR information memory portion 30 in apredetermined time, for example, during one second.

For this reason, the control portion 11 writes and stores (cashes) theimage data and the structure identification information corresponding tothe image data corresponding to the AR information acquisition request,and the AR information of the structure, the latitude and longitudeinformation, the azimuthal information and the angle of view informationin the buffer portion of the AR information memory portion 30 in apredetermined time, for example, during one second.

Moreover, when the hold control signal is input, immediately beforereceiving the hold control signal or at the time of AR informationacquisition request before the preset time, the control portion 11writes and stores the image data which is obtained from the informationretrieval system 2, stored in the buffer portion, and obtained for eachAR information acquisition request corresponding to the imageidentification information, the structure identification informationcorresponding to the image data, the AR information of the structure,the latitude and longitude information, the azimuthal information andthe angle of view information in the AR information memory portion 30,as the AR information table added with the time information.

Furthermore, the control portion 11 writes and stores the image datatogether with the image identification information on the memory portion15.

Moreover, even when the non-display mode setting the AR display as thenon-display is set, the control portion 11 and the hold control portion31 may perform the acquisition of the AR information or the holdprocessing of the AR information.

When the AR information of the held image is displayed by theabove-mentioned processing, the control portion 11 compares the timeinformation with the internal clock from the AR information table storedin the AR information memory portion 30 by the instruction signal basedon the press-down of the button (the AR switching button ARB2) of the ARdisplay confirmation provided in the case of the digital camera 1, orthe instruction signal based on the touch of the icon (the AR switchingbutton ARI2) of the AR display confirmation displayed on the displayportion 17, selects and reads the AR information table of the latesttime from the AR information memory portion 15, and displays the imagedata together with the tag of the AR information on the display portion17, similarly to upon being transmitted from the information retrievalsystem 2. In addition, the control portion 11 may periodically acquirethe AR information table of the latest time from the informationretrieval server 21 and may acquire the AR information table of thelatest time from the information retrieval server 21 based on theinstruction signal. Moreover, in the case of the AR informationacquisition mode, the control portion 11 may display the AR informationof the held image by receiving the hold control signal upon being held.

Furthermore, the control portion 11 may read the image datacorresponding to the image identification information stored in the ARinformation memory portion 30 from the memory portion 15 and display thethumbnail image of the read image data on the display portion 17. Thecontrol portion 11 may be configured so as to select the image data bytouching the thumbnail image displayed on the display portion 17 by auser. The control portion 11 reads the AR information table of the imageidentification information from the AR information memory portion 30 bythe image identification information corresponding to the selectedthumbnail image, thereby performing the display relative to the displayportion 17.

At this time, as mentioned above, a user can select the display /non-display of the AR information.

Herein, if the control portion 11 stores all of, the structureidentification information for identifying the structure, the structurename that is the name of the structure, the structure information, thepositional information by the latitude, the longitude or the like of thestructure, the description of the structure, and the submissioninformation, which are described in the structure table as the ARinformation, on the memory portion 15 or the like, even in a place wherethe wireless does not communicate with the wireless base station 3, thatis, even in the case of being off-line, the control portion 11 can addthe tag of the AR information to the image data stored in the ARinformation memory portion 30, as shown in FIG. 10 and can display theimage data on the display portion 17.

Furthermore, if it is not used while off-line, the control portion 11stores the latitude and longitude information of the digital camera 1,the azimuthal angle information, and the angle of view information whichare necessary for the AR information acquisition request as the ARinformation table, and when the image data is selected, the controlportion 11 may perform the AR information acquisition request withrespect to the information retrieval system again.

By the above-mentioned configuration, even when a user does not directthe imaging direction of the digital camera 1 in a direction wanted toobtain the AR information, for example, even in a state in which thescreen of the display portion 17 is viewed with mounting the digitalcamera 1 on the table, it is possible to confirm the AR informationwhile viewing the image data of a direction wanted to obtain the ARinformation.

Furthermore, in the case of the AR information acquisition mode, whenthe hold control signal is received from the hold control portion 31,the control portion 11 displays the display to prompt the input of theuser ID and the icon on which the keys of the alphabet and numbers areshown together with the user ID fill-up column.

A user touches the icon on which the keys of the alphabet and numbersare shown, whereby the control portion 11 detects the correspondingcharacter data, writes the same in the internal buffer, and displays thecharacter data of the internal buffer in the user ID fill-up column.

Moreover, the control portion 11 fixes the character row of the internalbuffer as the user ID by detecting the touch of the enter key that is inthe icon on which the keys of the alphabet and numbers are shown.

The control portion 11 adds and stores the user ID, which is stored inthe internal buffer, in the AR information table.

When a user tries to refer the AR information, if the image data isselected, the control portion 11 displays the user ID fill-up columntogether with the display of prompting the input of the user ID and theicon on which the keys of the alphabet and numbers are shown.

By detecting the touch of the icon on which the keys of the alphabet andnumbers are shown, the control portion 11 detects and writes thecorresponding character data on the internal buffer, and displays thecharacter data of the internal buffer in the user ID fill-up column.

Moreover, by detecting the touching of the enter key which is in theicon in which the keys of the alphabet and numbers are shown, thecontrol portion 11 compares the user ID corresponding to the characterrow of the internal buffer with the user ID added to the AR informationtable corresponding to the image identification information of theselected image data.

When the input user ID of a user is identical to the user ID added tothe AR information table, the control portion 11 displays thecomposition image (the AR image) of the image data and the ARinformation on the display portion 17, and on the other hand, when theinput user ID is different from the user ID added to the AR informationtable, the control portion 11 displays only the image data.

By the configuration, the digital camera 1 can specify the user (or theuser ID) for each image and can perform the display of the ARinformation.

Furthermore, when the AR information registered on the informationretrieval system 2 by another user or the AR information acquired byanother user is not displayed but a user wants to display the ARinformation registered by oneself or the AR information acquired byoneself, the control portion 11 houses a user who uses the digitalcamera 1 like an address book as the user table on the memory portion 15or the database 22 in advance, and when the icon (the AR switching iconARI2) to be referring the AR information is touched, the control portion11 displays the list (or the list of the user ID) of the user nameacquired from the user table on the display portion 17.

Moreover, when a user touches their user ID from the list, the controlportion 11 may read the AR information table, in which the same user asthe touched user ID is added, from the AR information memory portion 31,read the image data having the same image identification information asthat of the AR information table from the memory portion 15 and displaysthe image data on the display portion 17 as the thumbnail image. Inaddition, in the state in which the list is displayed, the controlportion 11 may read the AR information table, in which the same user asthe user ID of another user is added, from the AR information memoryportion 30 based on the signal that another user is selected, read theimage data having the same image identification information as that ofthe AR information table from the memory portion 15, and display theimaged data on the display portion 17 as the thumbnail image.

Furthermore, when the AR information table for each image identificationinformation is stored in the AR information memory portion 30, thecontrol portion 11 may add the AR information received from theinformation retrieval system 2 as well as the parameter or the imagingdate and time (the time information) connected to the AR information asthe AR related information with respect to the AR information, and storethem on the AR information table to correspond to the respectivestructure identification information.

In this manner, for example, when a user displays the image data of tenyears ago, the AR information and the AR related information on thedisplay portion 17, upon pressing down the icon (the AR date retrievalicon ARI3) of the AR retrieval date displayed on the display portion 17,the control portion 11 detects the press-down of the icon of the AR dateretrieval by the sensor, and displays the input column, in which thedate is input, together with the icon in which the keys of the alphabetand numbers are shown on the display portion 17.

In addition, when a user inputs the date in the input column by the iconin which the keys of the alphabet and numbers are shown, the controlportion 11 retrieves and extracts the AR information table, to which thesame date as that date is added, from the AR information memory portion30.

Moreover, the control portion 11 reads the image identificationinformation corresponding to the extracted AR information table, readsthe image data of the image identification information from the memoryportion 15, and displays the image data on the display portion 17 as thethumbnail image.

A user touches the thumbnail image displayed on the display portion 17,whereby the control portion 11 detects the touching of the thumbnailimage by the sensor of the display portion 17 and detects the imageidentification information of the touched and selected image data.

Moreover, the control portion 11 reads the image data corresponding tothe image identification information of the selected image data from thememory portion 15 and displays the image data on the display portion 17.

In addition, the control portion 11 reads the AR information tablecorresponding to the image identification information and displays thecomposition image (the AR image), in which the AR information and the ARrelated information are overlapped with the image data displayed inadvance, on the display portion 17.

By the above-mentioned configuration, a user can view the image data often years ago and the AR information of ten years ago of the structurewhich is in the image data via the display portion 17.

Furthermore, by touching the icon (the AR switching icon ARI2) of thecurrent AR information displayed on the display portion 17, the controlportion 11 detects whether the icon of the current AR informationdisplay is touched by the detection signal of the touch sensor andextracts the AR information table of the image identificationinformation of the image data from the AR information memory portion 30.

Moreover, the control portion 11 adds the latitude and longitudeinformation, the azimuthal information, and the angle of viewinformation when the AR information is acquired between the present andthe past (for ten years in the case of the present embodiment) from theAR information table, and transmits the AR information retrieval requestto the information retrieval system 2. In addition, the control portion11 may acquire all of the AR information between the present and thepast (for ten years in the case of the present embodiment) from theinformation retrieval system 2, from the AR information table.

When the information retrieval server 21 receives the AR informationretrieval request from the digital camera 1, the information retrievalserver 21 extracts the structure identification information of thestructure in the retrieval range obtained by the latitude and longitudeinformation, the azimuthal information and the angle of viewinformation, which are added to the AR information retrieval request,from the map data of the database 22.

Moreover, the information retrieval server 21 reads the AR informationof the structure corresponding to the structure identificationinformation from the structure table of the database 22 and transmitsthe AR information to the digital camera 1.

As a result, the control portion 11 displays the AR information of thecurrent existing structure with respect to the previous image data in anoverlapped manner.

By the configuration, it can be easier to confirm the transition of thestructure between the past and the present.

Furthermore, when the AR information at the time (e.g., ten years ago)of the imaging of the subject is reserved in the database 22 of theinformation retrieval server 21 in a date and time sequence or the like,the control portion 11 may acquire the AR information of the timeinformation and display it on the display portion 17 based on the timeinformation (e.g., the current, ten years ago, etc.) of the current andthe past. A user can trace from the present to the past and can view theAR information corresponding to the image data in a time series.

Furthermore, in the above-mentioned configuration in which the user IDis set for each the image identification information of the image data,by the configuration capable of adding a plurality of user ID, it ispossible to limit the disclosure of the AR information only among aplurality of friends (users). Furthermore, the user ID may be set foreach the structure identification information, so that management can beperformed for each the AR information of the structure. In this case, amode of performing the writing of the user ID is set, so that thecontrol portion 11 rushes (urges, persuades) the input of the user IDfor each structure.

Moreover, when none of the user ID is set, the open display, that is,the viewing person (the user) is not limited, but anyone (the user) canview.

Only when the user ID is added to the AR information table correspondingto the selected image data, the control portion 11 displays the inputcolumn of the user ID and the icon, on which the keys of the alphabetand numbers are shown, on the display portion 17.

As described above, the combinations of the display of the ARinformation using the user ID are the following five types.

(1) the current (or the previous) image data and the composition imageof the AR information of the structure in the current (or the previous)image data

A user ID is added and a limitation display of a displayable user

(2) the current (or the previous) image data and the composition imageof the AR information of the structure in the current (or the previous)image data

A user ID is not added and an open display in which all the users can bedisplayed

(3) the current (or the previous) image data and the composition imageof the AR information of the current structure which is in the retrievalrange of the current (or the previous) image data

A user ID is added and a limitation display of a displayable user

(4) the current (or the previous) image data and the composition imageof the AR information of the current structure which is in the retrievalrange of the current (or the previous) image data

A user ID is not added and an open display in which all the users can bedisplayed

(5) display (AR information non-display) of only the current (or theprevious) image data

Furthermore, the control portion 11 can switch the display of the ARinformation using the user ID by the signal based on the selection ofthe AR user switching icon ARI4.

Next, an operation of an information retrieval system of FIG. 9, whichis another embodiment of the present invention, will be described usingFIG. 13. FIG. 13 is a flowchart that shows an operation example of aninformation retrieval system according to another embodiment of thepresent invention.

When a user touches the icon (the AR information acquisition icon ARI1)of the AR information acquisition mode that is displayed on the displayportion 17, the control portion 11 detects the touching of the icon ofthe AR information acquisition mode and sets the digital camera 1 to theAR information acquisition mode.

The control portion 11 displays the image data continuously suppliedfrom the imaging portion 13 on the display portion 17 (step S31).

Moreover, the control portion 11 reads the latitude and longitudeinformation (the positional information) showing the coordinate valuesof the latitude and the longitude, respectively and the azimuthalinformation showing the azimuthal angle of the optical axis of thedigital camera 1 from each of the GPS 14 and the azimuthal sensor 16(step S32).

At this time, the control portion 21 obtains the angle of viewinformation in the digital camera 1 when performing the AR informationacquisition request with respect to the information retrieval system 2by reading the angle of view information from the focus distance and themagnification from the table in which the angle of view informationcorresponds to each combination of the focus distance and themagnification stored in the memory portion 15.

When the latitude and longitude information, the azimuthal information,and the angle of view information are obtained, the control portion 11gives the AR information acquisition request the camera identificationinformation, the latitude and longitude information, the azimuthalinformation and the angle of view information of the digital camera 1and transmits the information to the information retrieval system 2(step S33).

The information retrieval server 21 obtains the retrieval rangeretrieving the structure from the latitude and longitude information,the azimuthal information and the angle of view information suppliedfrom the digital camera 1, retrieves the structure within the retrievalrange from the map data of the database 22, and reads the AR informationfrom the structure table using the structure identification informationof the retrieved structure within the retrieval range (step S34).

Moreover, the information retrieval server 21 transmits the read ARinformation together with the structure identification information ofthe AR information to the digital camera 1 corresponding to the cameraidentification information.

Next, the control portion 11 calculates the coordinate positions of eachstructure in the display coordinate of the image data of the displayportion 17 from the latitude and longitude information, the azimuthalinformation, the angle of view information, the retrieval distance (thatis added to the AR information and is transmitted from the informationretrieval system 2), and the latitude and longitude information of thestructure included in the supplied AR information, and displays theimage data, and then, displays the AR information of each structure inthe image data as the tag (step S35).

Moreover, the control portion 11 determines whether or not the azimuthalangle supplied from the azimuth sensor 16 is changed in a predeterminedangle (distance) (step S36). That is, the control portion 11 obtains adifference between the azimuthal angle stored in the azimuthal anglememory portion and the azimuthal angle supplied from the azimuth sensor16 and compares the difference with the preset angle (threshold amountof angle).

At this time, the control portion 11 decides that the azimuthal angle ischanged when the difference is changed in (or beyond) a predeterminedangle (threshold amount of angle) and returns the processing to stepS31, on the other hand, the control portion 11 decides that theazimuthal angle is not changed when the difference is not changed in (orbeyond) a predetermined angle (threshold amount of angle)and progressesthe processing to step S37.

Next, the control portion 11 decides whether or not the hold controlsignal is supplied from the hold control portion 31 (step S37).

At this time, the control portion 11 progresses the processing to stepS38 when the hold control signal is input, on the other hand, when thehold control signal is not input, the control portion 11 returns theprocessing to step S36.

Herein, the hold control portion 31, for example, compares theacceleration data from the acceleration sensor with a threshold value ofthe preset acceleration data. When the acceleration data supplied fromthe acceleration sensor is equal to or more than the threshold value ofthe acceleration data, the hold control portion 31 creates and transmitsthe hold control signal to the control portion 11. Meanwhile, when theacceleration data supplied from the acceleration sensor is equal to orless than the threshold value of the acceleration data, the hold controlportion 31 does not perform a processing of transmitting the holdcontrol signal to the control portion 11.

Moreover, the control portion 11 writes and stores the latitude andlongitude information (the positional information) showing thecoordinate values of the latitude and the longitude, respectively andthe azimuthal information showing the azimuthal angle of the opticalaxis of the digital camera 1 from each of the GPS 14 and the azimuthalsensor 16 on the memory portion 15 (step S38). Then, the control portion11 progresses (returns) the processing to step S36.

In the case of the AR information acquisition mode, in any one of thedisplay or the non-display of the AR information, the flowchart of FIG.13 is repeated.

As mentioned above, according to the present embodiment, the imagingdirection of the digital camera 1 is changed, that is, the imagingdirection is changed in a direction different from the acquisitiondirection of the AR information, and the display portion 17 is moved toa position most easily viewed by a user, whereby the confirmation of theAR information can be performed in the image data at the time ofacquiring the AR information.

For example, it is possible to hold the AR information of the structurewhich is the image data from Tokyo station north entrance and confirmthe AR information of the structure of the image data from Tokyo stationnorth entrance on the table of the restaurant. Furthermore, in a statein which a user holds the AR information of the structure in a northdirection from the current position and directs the digital camera 1 ina direction (e.g., a direction of the ground or an eastern direction)different from the north direction after the hold, a user can confirmthe AR information of the structure of the held image data. Furthermore,the digital camera 1 in the present embodiment can perform theabove-mentioned processing even in the case of being off-line in whichthe communication with the wireless base station 3 is not performed.

Next, a navigation function of the digital camera 1 will be described.Firstly, the AR information acquisition mode is set by a user and theimage data and the AR information are displayed as the tag. Moreover,the same processing can be performed even in the AR display confirmationmode after holding the AR information.

When the icon of the navigation displayed on the display portion 17 istouched, the control portion 11 detects the touching of the icon of thenavigation by the touch sensor and starts up the application (thenavigation portion) of the navigation.

The control portion 11 displays the display of prompting the touching ofthe tag of the structure requiring the guidance on the display portion17.

When a user touches the tag of the structure requiring the guidance, thecontrol portion 11 detects the structure identification informationcorresponding to the tag of the touched structure, and reads the ARinformation table corresponding to the image identification informationof the displayed image data from the AR information memory portion 30.

Moreover, the control portion 11 reads the latitude and longitudeinformation of the structure from the AR information corresponding tothe detected structure identification information from the ARinformation table, adds the latitude and longitude information of thecurrent position of the digital camera 1 and the latitude and longitudeinformation of the structure, and transmits the information to theinformation retrieval system 2 as the navigation request.

When receiving the navigation request, the information retrieval server21 retrieves the shortest course (or a plurality of courses) between theadded latitude and longitude information of the digital camera 1 and thelatitude and longitude information of the structure from the mapdatabase stored in the database 22.

That is, the information retrieval server 21 extracts the nearestintersection from the coordinate value, where the digital camera 1 issituated, shown by the latitude and longitude information from the mapdata, and sets the intersection as a start intersection.

Similarly, the information retrieval server 21 extracts the nearestintersection from the coordinate value, where the structure is situated,shown by the latitude and longitude information from the map data, andsets the intersection as an end intersection.

Moreover, the information retrieval server 21 creates a combination of aplurality of road routes, which connects from the start intersection tothe end intersection, from a road network of the map data, selects theroute in which the distance of the respective combinations becomes theshortest, and for example, by an algorithm in a graph theory whicheffectively solves the existing shortest route problem such as aDijkstra method of extracting the shortest route, the informationretrieval server 21 extracts the combination of the road route in whichthe sum distance becomes the shortest.

Furthermore, the information retrieval server 21 transmits the imagedata of the map showing the road route from the start intersection tothe end intersection to the digital camera 1 in which the navigationrequest is performed. Herein, the information retrieval server 21creates the image data of the map by setting the display color of theguidance route (a road route connecting the start intersection to theend intersection) in the image data of the map to be different fromother roads.

The control portion 11 changes the image data of the received map to asize which can be displayed on the display portion 17, and displays theimage data of the map after being changed on the display portion 17.

Moreover, the control portion 11 disposes a mark showing its ownposition in a position on the image data of the map, which is displayedon the display portion 17, corresponding to the positions of thelatitude and the longitude acquired by the GPS 14 in an overlappedmanner. As a result, a user can receive the guidance service of theguidance from the start intersection to the end intersection whileconfirming the movement of its own mark.

Furthermore, the image data and the AR information of the structure ofthe image data may be transmitted from the digital camera 1 to anotherdigital camera of another user, thereby performing the sharing of theinformation with another user.

In addition, the image data and the AR information of the structure inthe image data may be uploaded from the digital camera 1 to a websitesuch as an SNS (Social Networking Service) so that the information isshared with the plurality of other digital cameras. In this case, byadding the user ID to the AR information table, only a user whorecognizes the user ID can read or add the AR information correspondingto the image data in the SNS.

Herein, the SNS server has an AR information sharing database in whichthe image data and the AR information are housed.

A user can add and register the user information from the digital camera1 to the image data that he wants to register and the AR informationcorresponding to the image data.

Herein, when a user touches the icon of the user registration displayedon the display portion 17, the control portion 11 detects the touchingof the icon by the touch sensor and displays the registration screen onthe display portion 17. In the registration screen, the thumbnail imageof the image data corresponding to the acquired AR information of auser, and the icon, on which the keys of the alphabet and numbers areshown, are included.

In the registration screen, a user selects the image data that he wantsto register from the thumbnail image, inputs the user ID relative to theimage data and the name (or the nick name) of the user himself, andtouches the enter icon, whereby the control portion 11 accesses to theSNS server and transmits the image data and the AR information togetherwith the registration request.

When the registration request is received, the SNS server creates thefile of the name (or the nick name) of the user himself, and writes andstores the image data and the AR information, which are required toregister, on the AR information sharing database. The file has the sameconfiguration as the AR information table of FIG. 11.

Moreover, when a user touches the icon of the AR information referencethat is displayed on the display portion 17 of the digital camera 1, thecontrol portion 11 detects the touching of the icon by the touch sensor,and displays the reference screen on the display portion 17. In thereference screen, the icon, on which the keys of the alphabet andnumbers are shown, is also included.

Next, in the registration screen, a user inputs the user name (or thenick name) that they want to refer and the user ID by touching the icon,on which the keys of the alphabet and numbers are shown, and touches theenter icon, whereby the control portion 11 accesses to the SNS server,and transmits the name (or the nick name) and the user ID together withthe reference request of the image data and the AR information.

When the reference request is received, the SNS server retrieves the ARinformation sharing database by the name (or the nick name) added to thereference request, and extracts the file of the same name (or the nickname).

Moreover, the SNS server reads the image identification information towhich the same user ID as the user ID received from the AR informationof the file is added, and the image data corresponding to the imageidentification information, and transmits the read AR information andthe image data to the digital camera 1.

As mentioned above, the control portion 11 displays the input image dataand the AR information on the display portion 17.

Next, when a user touches the icon of the AR information change that isdisplayed on the display portion 17 of the digital camera 1, the controlportion 11 detects the touching of the icon by the touch sensor, anddisplays the change screen on the display portion 17. In the changescreen, the thumbnail image of the image data corresponding to the readAR information, the information writing column and the icon, on whichthe keys of the alphabet and numbers are shown, are included.

Moreover, in the change screen, when a user selects the image data ofthe AR information that he wants to change from the thumbnail image, thecontrol portion 17 detects the selected image data by the touch sensor,displays the selected image data on the display portion 17, allows theimage data to correspond to the positions of the respective structureson the displayed image data, and displays the tag of the AR informationin an overlapped manner.

Moreover, a user touches and selects the tag of the AR information thathe wants to change, inputs the character row or the user ID which wereadded or edited to the information writing column by touching the keysof the alphabet and numbers, and touches the enter key (the enter icon),whereby the control portion 11 sets the character row input in theinformation writing column to the AR information of the new structure,and transmits the same together with the image identificationinformation and the structure identification information, and thechanged AR information to the SNS server as the change request includingthe name of the user and the user ID.

When the change request is received, the SNS server retrieves the ARinformation database by the name (or the nick name) added to the changerequest and extracts the file of the same name (or the nick name).

Moreover, the SNS server decides whether or not the user ID added to theimage identification information coincides with the user ID added to thereceived change request, and when they do not coincide with each other,the SNS server does nothing and notifies the digital camera 1 that itcannot change.

On the other hand, when the user ID added to the AR informationcoincides with the user ID added to the received change request, the SNSserver changes the AR information of the structure identificationinformation in the image identification information.

Furthermore, a program for realizing the function of the hold of the ARinformation of the control portion 11 and the hold control portion 31 ofthe digital camera 1 in FIG. 9 is recorded on a computer readablerecording medium, and the program recorded on the recording medium isread by the computer system and is executed, whereby the control of theAR information may be performed.

Furthermore, the program for realizing the function of the ARinformation retrieval of the information retrieval server 21 in FIG. 9may be recorded in the recording medium readable by a computer, and theprogram recorded on the recording medium may be read and executed on acomputer system, thereby performing the control of the retrieval of theAR information.

In addition, “the computer system” described herein includes hardwaresuch as an OS or a peripheral appliance.

Furthermore, in the case of using a WWW system, “the computer system”also includes a home page provision environment (or a displayenvironment).

Moreover, “the computer readable recording medium” refers to atransportable medium such as a flexible disk, an optical magnetic disk,a ROM, or a CD-ROM, or a memory device such as a hard disk built intothe computer system. Furthermore, “the computer readable recordingmedium” also includes a medium which dynamically stores the program in ashort time such as a communication line in the case of transmitting theprogram via a network such as the Internet or a communication circuitsuch as the telephone circuit, and a medium which stores a program for acertain time a nonvolatile memory in the computer system becoming aserver or a client in that case. Furthermore, the program may be one forrealizing a part of the above-mentioned functions, and may be one inwhich the above-mentioned functions can be realized by the combinationwith the program recorded in the computer system in advance.

As described above, the invention has been described with reference tothe drawings, but the specific configuration is not limited to theembodiment, and a design of a range without departing from the gist ofthe invention or the like is also included.

While embodiments of the invention have been described and illustratedabove, it should be understood that these are exemplary of the inventionand are not to be considered as limiting. Additions, omissions,substitutions, and other modifications can be made without departingfrom the spirit or scope of the invention. Accordingly, the invention isnot to be considered as being limited by the foregoing description, andis only limited by the scope of the appended claims.

What is claimed is:
 1. An imaging device comprising: an imaging portionthat images a subject; a positional information acquisition portion thatacquires positional information of an imaging position; a controlportion which acquires information on the subject based on thepositional information, and displays image data of the subject and theinformation on the subject on a display portion; and a hold controlportion that outputs a hold control signal, which holds the image dataof the subject and the information on the subject, to the controlportion.